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Abstract 

We introduce the problem of releasing sensitive data under differential privacy when the 
privacy level is subject to change over time. Existing work assumes that privacy level is de¬ 
termined by the system designer as a fixed value before sensitive data is released. For certain 
applications, however, users may wish to relax the privacy level for subsequent releases of the 
same data after either a re-evaluation of the privacy concerns or the need for better accuracy. 
Specifically, given a database containing sensitive data, we assume that a response yi that pre¬ 
serves ei-differential privacy has already been published. Then, the privacy level is relaxed to 
€2, with £2 > ei, and we wish to publish a more accurate response t/2 while the joint response 
{yi,y2) preserves e2"differential privacy. How much accuracy is lost in the scenario of gradu¬ 
ally releasing two responses yi and j/2 compared to the scenario of releasing a single response 
that is e2-differentially private? Our results show that there exists a composite mechanism that 
achieves no loss in accuracy. 

We consider the case in which the private data lies within M” with an adjacency relation 
induced by the £i-norm, and we focus on mechanisms that approximate identity queries. We 
show that the same accuracy can be achieved in the case of gradual release through a mechanism 
whose outputs can be described by a lazy Markov stochastic process. This stochastic process 
has a closed form expression and can be efficiently sampled. Our results are applicable beyond 
identity queries. To this end, we demonstrate that our results can be applied in several cases, 
including Google’s RAPPOR project, trading of sensitive data, and controlled transmission of 
private data in a social network. Finally, we conjecture that gradual release of data without 
performance loss is an intrinsic property of differential privacy and, thus, holds in more general 
settings. 
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1 Introduction 


Differential privacy is a framework that provides rigorous privacy guarantees for the release of 
sensitive data. The intrinsic trade-off between the privacy guarantees and accuracy of the privacy¬ 
preserving mechanism is controlled by the privacy level e G [0,oo); smaller values of e imply 
stronger privacy and less accuracy. Specifically, end users, who are interested in the output of the 
mechanism, demand acceptable accuracy of the privacy-preserving mechanism, whereas, owners of 
sensitive data are interested in strong enough privacy guarantees. 

Existing work on differential privacy assumes that the privacy level is determined prior to 
release of any data and remains constant throughout the life of the privacy-preserving mechanism. 
However, for certain applications, the privacy level may need to be revised after data has been 
released, due to either users’ need for improved accuracy or after owners’ re-evaluation of the 
privacy concerns. One such application is trading of private data, where the owners re-evaluate their 
privacy concerns after monetary payments. Specifically, the end users initially access private data 
under ei privacy guarantees and they later decide to “buy” more accurate data, relax privacy level 
to 62, and enjoy better accuracy. Furthermore, the need for more accurate responses may dictate 
a change in the privacy level. In particular, a database containing sensitive data is persistent over 
time; e.g. a database of health records contains the same patients with the same health history over 
several years. Future uses of the database may require better accuracy, especially, after a threat 
is suspected (e.g. virus spread, security breach). These two example applications share the same 
core questions. 

Is it possible to release a preliminary response with ei-privacy guarantees and, later, release a 
more accurate and less private response with overall e2-privacy guarantees? How is this scenario 
compared to publishing a single response under e2-privacy guarantees? In fact, is the performance 
of the second response damaged by the preliminary one? 

Composition theorems [T] provide a simple, but suboptimal, solution to gradually releasing 
sensitive data. Given an initial privacy level ei, a noisy, privacy-preserving response yi is generated. 
Later, the privacy level is increased to a new value 62 and a new response 2/2 is published. For an 
overall privacy level of 62, the second response 2/2 needs to be (62 — ei)-private, according to the 
composition theorem. Therefore, the accuracy of the second response deteriorates because of the 
initial release yi. 

In this work, we derive a composite mechanism which exhibits no loss in accuracy after the 
privacy level is relaxed. This mechanism employs correlation between successive responses, and, to 
the best of our knowledge, is the hrst mechanism that performs gradual release of sensitive data. 

1.1 Our Results 

This work introduces the problem of gradually releasing sensitive data. Our results focus on the 
case of vector-valued sensitive data u G M"' with an £i-norm adjacency relation. Our first result 
states that, for the one-dimensional (n = 1) identity query, there is an algorithm which relaxes 
privacy in two steps without sacrificing any accuracy. Although our technical treatment focuses on 
identical queries, our results are applicable to a broader family of queries. We also prove the Markov 
property for this algorithm and, thus, we can easily (without any computational complexity) relax 
privacy in any number of steps. These two results provide a different perspective of differential 
privacy, and lead to the definition of a lazy Markov stochastic process indexed by the privacy level 
6. Gradually releasing sensitive data is performed by sampling once from this stochastic process. 
We also extend the results to the high-dimensional case. 

On a theoretical level, our contributions add a whole new dimension to differential privacy — 
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that of a varying parameter e. We focus on the mechanism that adds Laplace-distributed noise 14 
to the private data u G M"": 



QeU = U + 



( 1 ) 


where e is the privacy level, || • ||i is the £i-norm, and are independent and identically 

distributed samples from the stochastic process {14}e>o which has the following properties: 

1 . { 14 }e>o is Markov: 14^-1143 1142 , for any 63 > 62 > ei > 0 . 

2 . 14 is Laplace-distributed: P (14 = x) = 

3 . { 14 }e>o is lazy, i.e. there is positive probability of not changing value): 

P( 14 i = x |142 =y) = -y) + ^ 1 - ^ where €2 > ei > 0 , 

where 6 is Dirac’s delta function. For a hxed e, mechanism 0 reduces to the Laplace 
mechanism. 

Mechanism 0 has the following properties and, thus, performs gradual release of private data: 

• Privacy: For any set of privacy levels the mechanism that responds with {QeiU}'^i 

is (max™ ^ ej)-private. 

• Accuracy: For a fixed e, the mechanism is the optimal e-private mechanism. 

In practice, gradual release of private data is achieved by sampling the stochastic process { 14 }e>o: 

1 . Draw a single sample {ue}e>o from the stochastic process { 14 }£>o. 

2 . Compute the signal y^ = u + v^:, e > 0 . 

3 . For ei-privacy guarantees, release the random variable y^^- 

4 . Once privacy level is relaxed from ei to 62, where £2 > ei, release the random variable yej- 

5 . In order to relax privacy level in an arbitrarily many times, ei —t' 62 e^, repeat the 

last step. 

More formally, our main result derives a composite mechanism that gradually releases private 
data by relaxing the privacy level in an arbitrary number of steps. 

Theorem 1 (A. Gradual Privacy as a Composite Mechanism). Let M"' be the space of privacy 
data equipped with an ii-norm adjacency relation. Consider m privacy levels such that 

0 < ei < • • • < Cm which successively relax the privacy level. Then, there exists a composite 
mechanism Q of the form 


Qu:= {u + Vi,...,u + Vm ), 


( 2 ) 


such that: 
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Figure 1 : Gradual release of identity queries is achieved with the use of the stochastic process Ve for e > 0 . 
For tight values of privacy (e —)■ 0 ), high values of noise (|tan“^ f) returned, whereas, almost 

zero samples (14 —)■ 0 ) are returned for large privacy budgets (e —>■ oo). The process Ve is Markov; future 
samples depend only on the current value of the process which eases implementation. Furthermore, the 
process is lazy; the value of the process changes only a few times. 


L The restriction of the mechanism Q to the first j coordinates {u + Vi, ... ,u + Vj) is ej-private, 
for any j G {1,.. .,m}. 

2. Each coordinate j G m} of the mechanism u -\-Vj achieves the optimal mean-squared 

error E||l^'H^. 

The mechanism that satisfies Theorem has a closed-form expression and provides a new 
perspective of differential privacy. Instead of designing composite mechanisms of the form Q, we 
consider the continuum of privacy levels e G [0, oo). Our results are more succinctly stated in terms 
of a stochastic process {I4}e>o. A composite mechanism is recovered from the stochastic process 
by sampling the process {Ve}e>o at a finite set of privacy levels 

Theorem 1 (B. Gradual Privacy as a Stochastic Process). Let M”" be the space of privacy data 
equipped with the ii-norm. Then, there exists a stochastic process {I4}e>o} that defines the family 
of mechanisms parametrized by e: 


Qeu:=u + Vf_, eG(0, oo), 


(3) 


such that: 

• Privacy: For any e > 0, the mechanism that releases the signal {u + Pcr}o-g(o,e] e-private. 

• Accuracy: The mechanism that releases the random variable u + 14 is the optimal e- 
private mechanism, i.e. the noise sample 14 achieves the optimal mean-squared error E|| 14Hi- 

From a more practical point of view, our results are applicable to cases beyond identity queries. 
Specifically, our results are directly applicable to a broad family of privacy-preserving mechanisms 
that are built upon the Laplace mechanism and, informally, have the following form. The sensitive 
data is initially preprocessed, then, the Laplace mechanism is invoked, and, finally, a post-processing 
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step occurs. Under the assumption that the preprocessing step is invariant of the privacy level, 
gradual release of sensitive data is possible. We demonstrate the applicability of our results on 
Google’s RAPPOR project [2], which analyzes software features that individuals use while respect¬ 
ing their privacy. In particular, if a software feature is suspected to be malicious, privacy level 
can be gradually relaxed and a more accurate analysis can be performed. On another direction, 
our results broaden the spectrum of applications of differential privacy. To this end, we present 
an application to social networks where users have different privacy concerns against close friends, 
acquaintances, and strangers. 

We conclude our paper with a conjecture. Although present work focuses on mechanisms that 
add Laplace-distributed noise, we conjecture that the feasibility of gradually releasing sensitive 
data is a more general property of differential privacy. In particular, we formulate the conjecture 
that repeatedly relaxing the privacy level without loss of accuracy is possible for a larger family of 
privacy-aware mechanisms. 

1.2 Previous Work 

Differential privacy is an active field of research and a rich spectrum of differential private mech¬ 
anisms has appeared in the literature. The exponential mechanism [T] is a powerful and generic 
tool for building differential private mechanisms. In particular, mechanisms that efficiently approx¬ 
imate linear (counting) queries have received a lot of attention [3], [1], [5]. Besides counting queries, 
privacy-aware versions of more complex quantities have been introduced such as signal filtering [6] , 
optimization problems and allocation problems [H]. In addition to the theoretical work, 

differential privacy has been deployed in software tools [lOj . 

The aforementioned work assumes that the privacy level e is a designer’s choice that is held fixed 
throughout the life of the privacy-aware mechanism. To the best of our knowledge, our work is the 
first approach that considers privacy-aware mechanisms with a varying privacy level e. Gradually 
releasing private data resembles the setting of differential privacy under continuous observation, 
which was first studied in m- In that setting of the privacy level remains fixed while more 
sensitive data is being added to the database and more responses are released. In contrast, our 
setting assumes that both the sensitive data and the quantity of interest are fixed and the privacy 
level e is varying. 

Gradual release of sensitive data is closely related to optimality results. Work in |3] established 
optimality results in an asymptotic sense (with the size of the database). Instead, our work requires 
exact optimality results and, therefore, is presented within a tighter version of differential privacy 
that was explored in m, [13], where exact optimality results exist. This tighter notion which is 
targeted for metric spaces and we call Lipschitz privacy, allows for the use of optimization techniques 
and calculus tools. Prior work on Lipschitz privacy includes the exact optimality of the Laplace 
mechanism is established under Lipschitz privacy m, [Hi- 

On a more technical level, most prior work on differential privacy 0,0,0 introduces differen¬ 
tial private mechanisms that are built upon the Laplace mechanism and variations of it. Although 
building upon the Laplace mechanism limits the solution space, there is a good reason for doing so. 
Specifically, for non-trivial applications, the space of probability measures can be extremely rich 
and hard to deal with. Technically, our approach deviates from prior work by searching over the 
whole space of differential private mechanisms. Work in m is another example that proposes a 
non-Laplace distribution in order to achieve better performance on subsequent queries while satis¬ 
fying overall differential privacy constraints. The Laplace mechanism, then, naturally emerges as 
the optimal mechanism. 
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2 Background Information 

2.1 Differential Privacy 

The framework of differential privacy [16j , m dictates that, whenever sensitive data is accessed, a 
noisy response is returned. The statistics of the injected noise are deliberately designed to ensure 
two things. First, an adversary that observes the noisy response cannot confidently infer the original 
sensitive data. The privacy level is parametrized by e G [0,oo), where smaller values of e imply 
stronger privacy guarantees. Second, the noisy response can still be used as a surrogate of the exact 
response without severe performance degradation. On the other hand, the accuracy of the noisy 
response is quantified by the mean-squared error from the exact response. 

Work in m defined differential privacy, which provides strong privacy guarantees against a 
powerful adversary. 

Definition 2 (Differential Privacy). LetlA he a set of private data, A T be a symmetric binary 
relation (called adjacency relation) andy be a set of possible responses. Fore > 0, the randomized 
mapping Q : Id ^ A (T) (called mechanism) is e-differentially private if 

F{Qu G 5) < eT(Qu' G 5), V(«, u') G .4, V5 C T. (4) 

Remark 1. We assume the existence of a rich-enough a-algebra M <Z 2^ on the set of possible 
responses y. Then, A (T) denotes the set of probability measures over {M,y). 

Let y ~ Qu be a noisy response produced by the e-differentially private mechanism Q. For 
brevity, we say that “output y preserves e-privacy of the input n”. 

The adjacency relation A captures the aspects of the private data u that are deemed sensitive. 
Consider a scheme with n users, where each user i contributes her real-valued private data Ui G M, 
and a private database u = [ui, ... ,Un] G M" is composed. For a > 0, an adjacency relation that 
captures the participation of a single individual to the aggregating scheme is defined as: 

{u,u') G Ahq s.t. Ui = Ui,\li A j and \uj — u'j\ < a. (5) 

Adjacency relation can be relaxed to Ai^, which is induced by the £i-norm and is defined as: 

(u,«') G .4^^ ||tt — rt||i < a, (6) 

where it holds that Ai^ C At,.^. 

Resilience to post-processing establishes that any post-processing on the output of an e-differentially 
private mechanism cannot hurt the privacy guarantees. 

Proposition 3 (Resilience to Post-Processing). Let Q : Id ^ A (T) he an e-differentially private 
mechanism and g : y ^ Z he a possibly randomized function. Then, the mechanism g o Q is also 
e-differentially private. 

More complicated mechanisms can be defined from simple ones using the composition theorem. 

Proposition 4 (Composition). Let mechanisms Qi,Q 2 ■ Id —)■ A (T) respectively satisfy ei and 
e 2 -differential privacy. Then, the composite mechanism Q : Id ^ A (T^) defined by Q = {Qi,Q 2 ) 
is (ei -|- e 2 )-differentially private. 

Proposition [^provides privacy guarantees whenever the same sensitive data is repeatedly used. 
Moreover, the resulting privacy level ei -|- e 2 given by Proposition is an upper bound and can 
severely over-estimate the actual privacy level. The mechanism presented in this paper introduces 
correlation between mechanisms Qi and <52; so that it provides much stronger privacy guarantees. 
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2.2 Lipschitz Privacy 

Lipschitz privacy na, m is a slightly stronger version of differential privacy and is often used 
when the data is dehned on metric spaces. 

Definition 5 (Lipschitz Privacy). Let {U, d) he a metric space and 3^ be the set of possible responses. 
For € > 0, the mechanism Q is e-Lipschitz private if the following Lipschitz condition holds: 

|lnP(Qtt G 5) — lnP(Qu G 5)1 < ed(M, tt'), \/u,u £ U, \/S C y. (7) 

Lipschitz privacy is closely related to the original definition of differential privacy, where the 
adjacency relation A in differential privacy is defined through the metric d. In fact, any Lipschitz 
private mechanism is also differentially private. 

Proposition 6. For any a > 0, an e-Lipschitz private mechanism Q is ae-differentially private 
under the adjacency relation A: 

{u, u') £ A d{u, u') < a. (8) 

Adjacency relation defined in (|^ can be captured by the .^i-norm under the notion of 
Lipschitz privacy; the metric d is d{u,u') = ||u — u'\\i. 

Our results are stated within the Lipschitz privacy framework. Proposition implies that our 
privacy results remain valid within the framework of differential privacy. For brevity, we call an 
e-Lipschitz private mechanism as e-private and imply that a differentially private mechanism can 
be derived. 

Similar to differential privacy, Lipschitz privacy is preserved under post-processing (Proposi¬ 
tion]^ and composition of mechanisms (Proposition]^. Compared to differential privacy, Lipschitz 
privacy is more convenient to work with when the data and adjacency relation are dehned on a 
metric space, which allows for the use of calculus tools. Under mild assumptions, the Lipschitz 
constraint (]^ is equivalent to a derivative bound. In particular, for U = M” equipped with the 
metric induced by the norm || • ||, a mechanism Q is e-Lipschitz private if 

||V„lnP(Qu = y)||, <e, (9) 

where || • ||* is the dual norm of || • ||. In practice, we check condition to establish the privacy 
properties of mechanism Q. 

2.3 Optimality of the Laplace Mechanism 

Computing the optimal private mechanism for a hxed privacy level e is considered an open problem. 
Specihcally, let IL be the space of private data, A be an adjacency relation, q : IL ^ y he a query, 
and e be a hxed privacy level. The exponential mechanism [T] is a popular technique for constructing 
private mechanisms. 

Proposition 7 (Exponential Mechanism). Let s : IL x y ^ M. be 1-Lipschitz in {U,d). Consider 
the mechanism Q whose output satisfies 

F{Qu = y) (X (10) 


Then, Q is e-Lipschitz private. 

The Laplace mechanism is a special instance of the exponential mechanism for real spaces 

{FAJi). 
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Figure 2: The staircase mechanism is the optimal e-differential private mechanism, whereas the Laplace 
mechanism is the optimal e-Lipschitz private mechanism. Therefore, the 


Definition 8 (Laplace Mechanism). Let be the space of private data. The Laplace mech¬ 

anism is defined as: 

Qu = u-\-V, where V ~ (11) 

The Laplace mechanism can be shown to be e-differentially private. In general, however, the 
Laplace mechanism is suboptimal in the sense of minimum mean-squared error. For the single¬ 
dimensional case, the staircase mechanism [18] is the optimal e-differentially private mechanism; 
the mechanism which adds noise V whose distribution is shown in Figure]^ However, the Laplace 
mechanism is proven to be the optimal e-Lipschitz private mechanism in the sense of both minimum 
entropy [ 13 ] and minimum mean-squared error m, whereas the staircase mechanism fails to satisfy 
Lipschitz privacy due to its discontinuous probability density function. 

Theorem 9 l |13j Optimality of Laplace). Consider the e-Lipschitz private (in (M"^, (.i)) mechanism 
Q : M"" —)■ A (M"") of the form Qu = u-\-V, with V ~ giV) G A (M"^). Then, the Laplace mechanism 
that adds noise with density li{v) = (1 )”minimizes the mean-squared error. Namely, for 
any density g, we have: 

E\\Qn - u||i = E ||Hf > E ||H||i = ( 12 ) 

V'^g 6 

The optimal private mechanism characterizes the privacy-performance trade-off and is required 
for gradually releasing sensitive data. Thus, optimality of the Laplace mechanism in Theorem]^ is 
a key ingredient in our results and renders the problem tractable. 

3 Gradual Release of Private Data 

The problem of gradually releasing private data is now formulated. Initially, we focus on a single 
privacy level relaxation from to ei to 62 and a single-dimensional space of private data U = M. 
Subsections |3.2| and |3.3| present extensions to high-dimensional spaces and multiple rounds of privacy 
level relaxations, respectively. 

Consider two privacy levels ei and €2 with €2 > ei > 0. We wish to design a composite 
mechanism : IT ^ A {y x y) that performs gradual release of data. The first and second 
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coordinates respectively refer to the initial ei-private and the snbsequent e 2 -private responses. In 
practice, given privacy levels ei and 62 and an input u gU, we sample (yi, ^ 2 ) from the distribution 
Initially, only coordinate yi is published satisfying ei-privacy guarantees. Once privacy 
level is relaxed to 62 , response 1/2 is released as a more accurate response of the same query on the 
same private data. 

An adversary that wishes to infer the private input u eventually has access to both responses yi 
and y 2 - Therefore, the pair (^ 1 , 2 / 2 ) needs to satisfy e 2 -privacy. On the other hand, an honest user 
wishes to maximize the accuracy of the response and, therefore, she is tempted to use an estimator 
Um = ^( 2 / 1 , 2 / 2 ) and infer a more accurate response yM- In order to relieve honest users from any 
computational burden, we wish the best estimator to be as the truncation: 


0{yi,y2) = 2 / 2 - 


(13) 


The composition theorem pQ provides a trivial, yet highly conservative, approach. Specifically, 
compositional rules imply that, if yi satisfies ei-privacy and ( 221 , 2 / 2 ) satisfies e 2 -privacy, coordinate 
2/2 itself should be (e 2 — ei)-private. In the extreme case that 62 — ei = <C 1, response 2/2 alone 
is expected to be (i-private and, therefore, is highly corrupted by noise. This is unacceptable, since 


estimator (13) yields an even noisier response than the initial response 2 / 1 - Even if honest users 


are expected to compute more complex estimators than the truncation one in (13), the approach 
dictated by composition theorem can still be unsatisfactory. 

Specifically, consider the following two scenarios: 

1. An ei-private response 2/1 is initially released. Once privacy level is relaxed from ei to € 2 , an 
supplementary response 2/2 is released. 


2. No response is initially released. Response y 2 is released as soon as the privacy level is relaxed 
to € 2 . 

Then, there is no guarantee that the best estimator 0 ( 2 / 1 , 222 ) in Scenariowill match the accuracy 
of the response y 2 in Scenario An accuracy gap between the two scenarios would severely impact 
differential privacy. Specifically, the system designer needs to be strategic when choosing a privacy 
level. Differently stated, a market of private data based on composition theorems would exhibit 
friction. 

The key idea to overcome this friction is to introduce correlation between responses 2/1 and 2 / 2 - 
In this work, we focus on Euclidean spaces U = M” and mechanisms Qu = u + V that approximate 
the identity query q{u) = u. Our main result states that a frictionless market of private data is 
feasible and Scenarios and are equivalent. This result has multi-fold implications: 

• A system designer is not required to be strategic with the choice of the privacy level. Specif¬ 
ically, she can initially under-estimate the required privacy level with ei and she can later 
fine-tune it to €2 without hurting the accuracy of the final response. 


• A privacy data market can exist and private data can be traded “ 62 / the pound". An ei- 
private response 2/1 can be initially purchased. Next, a supplementary payment can be made 
in return for a privacy level relaxation to €2 and a refined response 2 / 2 - The accuracy of the 
refined response 2/2 is, then, unaffected by the initial transaction and is controlled only by the 
final privacy level € 2 - 


More concretely, given privacy levels ei and €2 with 62 > ei, we wish to design a composite 
mechanism : U ^ y x y with the following properties: 




1. The restriction of to the first coordinate should match the performance of the optimal 

ei-private mechanism . More restrictively, the first coordinate of the composite mechanism 

should be distributed identically to the optimal ei-private mechanism 

P {Qei^e2U G 5 X T) = P (QeiU G S), G U and S <Gy (14) 

2. The restriction of to the first coordinate should be ei-private. This property is imposed 

by constraint 

3. The restriction of to the second coordinate should match the performance of the 

optimal e 2 -private mechanism Similarly to the first coordinate, the second coordinate 
of the composite mechanism must be distributed identically to the optimal e 2 -private 

mechanism 

P G T X 5) = P {Qe 2 U G S) ,\/u GU and 5 C T (15) 

4. Once both coordinates are published, e 2 -privacy should be guaranteed. According to Lipschitz 
privacy, the requirement is stated as follows: 

P (Qe^^ejU G S) is € 2 -Lipschitz in u, for all S C y'^. (16) 


Equations (14) and (15) require knowledge of the optimal e-private mechanism. In general, 
computing the e-private mechanism that maximizes a reasonable performance criterion is still an 
open problem. Theorem [^establish the optimality of the Laplace mechanism as the optimal private 
approximation of the identity query. 


3.1 Single-Dimensional Case 

Initially, we consider the single-dimensional case where U = M. equipped with the absolute value. 
Theorem establish the optimal e-private mechanism that is required by Equations (14) and (15): 


QeU = u + V, where E ~ e 


(17) 


Mechanism (0 minimizes the mean-squared error from the identity query among all e-private 
mechanisms that use additive noise: 


E (QeU — uY 
y~e-'|xi 


(18) 


Theorem 10 establishes the existence of a composite mechanism that relaxes privacy from ei to e 2 
without any loss of performance. 

Theorem 10. Consider privacy levels ei and €2 with e 2 > ei > 0, and mechanisms of the form: 


Qiu := M -|- El and Q 2 U := u + V 2 , with (Ei, E 2 ) ^ g G A (M^) . 
Then, for density lei ,€2 with: 


^ei,e2ix,y) = p-e ^^^^^5{x-y) + 


ei(e 2 - (^1) ei\x-y\-e2\y\ 

4eo 


(19) 


( 20 ) 


where 5 is the Dirae delta function, the following properties hold: 
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1. The mechanism Qi is ei-private. 

2. The mechanism Qi is optimal, i.e. Q\ minimizes the mean-squared error KV^. 

3. The mechanism {Qi,Q 2 ) is e 2 -private. 

4- The mechanism Q 2 is optimal, i.e. Q 2 minimizes the mean-squared error ¥,¥2 ■ 

Proof. Consider the mechanism Q = {Qi,Q 2 ) induced by the noise density ( [20l ). We prove that 
this mechanism satisfies all the desired properties: 

1. The first coordinate is Laplace-distributed with parameter For x > 0, we get: 


P(t/^=a:)= [ 5(x,y)dy = ^ f ^-e,\x-y\-e2\y\^y 

Jm. 2e2 462 Jm 

^2 ^ ^2\ / rO 

= ^1(^2 ~ gj) 

262 462 


( /*0 px 

/ g-ei^+(<:i+':2)y^y _j_ / ^-<^ix-{e2-€i)y^y 

J —00 J 0 




262^ ' ^ 


£1(^2 (-l) eix {ei+e 2 )y ^ _ ^l(^l + ^2) -eix „-{e 2 -ei)y 


( 21 ) 


462 


462 


_ ei{e 2 - 61 ) (ei+e 2 )y 

462 


ei _ 
= —e 

2 


EIX 


The case x < 0 follows from the symmetry (x, y) —>■ (—x, —y). Therefore, the first coordinate 
is 6 i-private and achieves optimal performance. 


2. The second coordinate is Laplace-distributed with parameter —. We have: 


F{V 2 = y)= I 5 (x,y)dx = + !i(!i- ^jlg-e 2 \y\ f 

Jr 


262 


462 


^\^-y\dy 


262® 


462 


da 


_^p-e2|y| 


( 22 ) 


= TTe-^2hl + ^ 

262 262 
= ^g-e2|y| 

2 


Thus, the second coordinate achieves optimal performance. 


3. Lastly, we need to prove that the composite mechanism is 62 -private. We handle the delta 
part separately by defining D = {x : (x,x) G S} for a measurable S C M^. The probability 
of landing in set S is: 


F{Qu G S) 


262 Jd 462 JJs 


(23) 
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We take the derivative and use Fubini’s theorem to exchange the derivative with the integral: 


j 2 

—¥{Qu G 5 ) = — / e2Sgn(x — 
du 262 Jd 


du 


+ 




462 


JJ 62Sgn(7/— tt)e '^^dxdy 


u€S) 


~ 262 Jd '''' 


- 62 | x -«|^^ + - ef). ff ^^^-ei\{x-u)-{y-u)\-e2\y-u\^^^y 

462 JJs 


—F{Qu G S) 
du 


< 62 P(Qn G 5) 


— lnP(QM G S) 
du 


< £2 


This completes the proof. 


( 24 ) 

□ 


3.1.1 Single Round of Privacy Relaxation 

Theorem | 10 | achieves gradual release of sensitive data in two steps, first with 6i-privacy and, then, 


with 62-privacy. In practice, Theorem 10 can be used as follows: 


Given the private value u G M, sample noise Pi ~ e and release response yi = u -|- Pi, 

which is optimal and respects 6i-privacy. 

Once privacy level is relaxed from 6i to 62, sample noise V2 from the conditioned on Pi 
distribution: 

2 2 

P(P2 = y\Vi =x) = — -x)+ ^2 - ^-er\y-x\-e2\y\+e^\x\^ ("25) 

£2 262 


and release response y2 = U + P2. Distribution ( 25 ) is derived from the joint distribution ( 20 ) 


and ensures both that {yi,y2) is 62-private and that P2 is optimally distributed. 


Conditional distribution ( 25 ) is shown in Figure]^ Note that for 62 = 61, Distribution ( 25 ) is 
reduced to a delta function: 


nV2 = y\Vi = x) = 6 {x - y). 


(26) 

thus, no updated response 


In words, for 62 = 61 no privacy relaxation effectively happens and 
is practically released. Moreover, for 62 —)• 00, a limiting argument shows that Distribution 25 
reduced to: 


IS 


P(P2 = y\Vi=x) = 5 {y). 


(27) 


Practically, letting 62 —)• 00 cancel any privacy constraints and the exact value of private data u 
can be released 1/2 = u- For general values of 61 and 62, Pearson’s correlation coefficient decreases 
for more aggressive privacy relations, pvi,V2 — Algorithm provides a simple and efficient way 
to sample P2 given Pi. 
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Figure 3: Gradual release of private data is performed in the following way. First, the ei-private response 
j/i = u + Fl is released, where Vi ~ Once privacy level is relaxed from ei = 1 to e 2 = 2, the 

supplementary response y2 = u + V2 is released, where F 2 is distributed as shown above. The composite 
mechanism that releases (j/i, j/ 2 ) is e 2 -private and V 2 is optimally distributed. 


Algorithm 1 Sampling from Distribution (25) for the second noise sample V 2 = y given the first 


noise sample Vi = x can be efficiently performed. 


Require: Privacy levels ei and € 2 , such that €2 > ei > 0, and noise sample x. 
function RelaxPrivacy(x, ei, € 2 ) 
switch randomly 

case with probability 



draw z 


case with probability 


for z < 0 

0, otherwise, 

return y = sgn(x)z. 

case with probability (l — : 

g-(e 2 -ei) 2 ^ for 0 < z < \x\ 

0, otherwise. 

y = sgn(x)z. 

case with probability e 


draw z 


draw z 


^2-^1 „-(e?-e:)\x\. 
2€2 

g-{ei+€2)z 


0 , 


for z > \x\ 
otherwise. 


return y = sgn(x)z. 

end switch 
end function 
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3.1.2 Single Round of Privacy Tightening 


Tightening the privacy level is impossible, since it implies revoking already released data. Nonethe¬ 
less, generating a more private version of the same data is still useful in cases such as private data 
trading. In that case, distribution ( 20 ) can be sampled in the opposite direction. Specifically, noise 
V2 is initially sampled, V2 ~ and the e2-private response 1/2 = u + V2 is released. Next, 

private data u is traded to a different agent under the stronger ei-privacy guarantees. Noise sample 
Vi is drawn from distribution 


P(Pi = x\V 2 = y)= ( 1 ) ' <5(0; - y) + (^1 - , (28) 

and the ei-private response yi = u -|- Vi is released. Remarkably, response yi can be generated 
conditioning only on y2‘. 


y2 = yi + 


( 29 ) 


where V2->i = Ri — V2 is independent of V2) R2->i-LV2- In words, tightening privacy under the 
Laplace mechanism does not require access to the original data u and can be performed by an 
agent other than the private data owner. Theorem suggests that the randomized post-processing 
2/1 = 2/2 + R2->'1 of the e2-private response y2 is at least e2-private. For V2^i given by distribution 
( 28 ), this tightening of privacy level is precisely quantified, i.e. 62 —>■ ei. Recall that our results are 
tight] no excessive accuracy is sacrificed in the process. 


3.2 High-Dimensional Case 

Theorem | 10 | can be generalized for the case that the space of private data is Euclidean M” equipped 
with the £i-norm. Theorem establishes that the Laplace mechanism: 

QeU = u + V, where V ~ ( 30 ) 

minimizes the mean-squared error from the identity query among all e-private mechanisms that use 
additive noise R G M": 


( 31 ) 

Theoremshows that each coordinate of R is independently sampled. This observation implies 
that Theorem can be applied to n dimensions independently. 

Theorem 11. Consider privacy levels ei, 62 with €2 > ei > 0. Let Qi be an ei-private mechanism 
and Q2 an €2-private mechanism of the form: 

Qiu := u + Vi and Q2U := n -|- R2 with (Ri, R2) ~ y G A (M^”) , ( 32 ) 

where u G Then, gradual release of sensitive data u from ei to 62 is achieved by the probability 
distribution C .' 

Cl jCJ 


C...(Ri,R2) = ( 33 ) 

i=l 


where R = [r/^\ ..., r/”^], i = 1 , 2 . Namely: 
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• Mechanism Qi is ei-private and optimal. 

• Mechanism Q 2 is the optimal € 2 -private mechanism. 

• Mechanism {Qi,Q 2 ) is € 2 -private. 

Proof. Let ..., denote the coordinates of a vector x G M"’. The desired probability 
distribution is defined by independently sampling each coordinate using Theorem 10 Let: 


= 9{x,y) = 


2 = 1 


The probability distribution satisfies the required marginal distributions: 


(34) 


9{x,y)d^y = 


ei 


and 


g{x,y)d^x = 


€2 


Moreover, it satisfies e 2 -privacy constraints: 


||V„lnP(( 5 iu = zi and Q 2 U = Z 2 )\\ 
= \\^u'xa.lf^^^^{zx-u,Z2-u)\\^ 

^ -hllf^^^^{zi -U,Z2- u) 


= max 

l<2<n 


= max 

l<2<n 


dui 


< max e 2 = 62 , 
l<2<r2 


where in the last line we used the fact that le^,e 2 is e 2 -private. This completes the proof. 


□ 


3.3 Multiple Privacy Relaxations 

Theorems 1 10| and 1 11 1 perform privacy relaxation from ei to 62 - However, the privacy level is possibly 
updated multiple times. Theorem 1 12 1 handles the case where the privacy level is successively relaxed 
from ei to € 2 , to 63 , until Cm- Specifically, Theorem [T^ enables the use of Theorem multiple 
times while relaxing privacy level from a to Cj+i for i G {1,..., m — 1}. We call this statement the 
Markov property of the Laplace mechanism. 

Theorem 12. Consider m privacy levels {ei}T with 0 < ei < • • • < and mechanisms Qi of 
the form: 


QiU = u + Vi, with (Hi,..., Vm) ~ 5 G A (M”*). 
Consider the distribution g = with: 


m—1 


■ ■ ■ -iVm) — leiivi) 


2=1 


lti,ei+i jVi: Vj+l) 
Ci {Vi) 


(35) 


( 36 ) 


where le{v) = §e Then, distribution 4i,...,e„i has the following properties: 
1. Each prefix mechanism {Qi,... ,Qi) is ei-private, for i G {!,..., m}. 
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2. Each mechanism Qi is the optimal Ci-private mechanism, i.e. it minimizes the mean-squared 
error . 

Proof. The proof uses induction on m. The case m = 2 is handled by Theorem For brevity, 
we prove the statement for m = 3. Let f{x,y) = lei,e 2 ix,y) and g{y,z) = Ie 2 ,e 3 {y,z). Consider the 
joint probability 


Hx,y,z) = U^^e2,e3{x,y,z) 


f{x,y)g{y,z) 

h2{y) 


(37) 


where le 2 {y) = Measure h possesses all the properties that perform gradual release of 

private data: 

• All marginal distributions of measure h are Laplace with parameters and respec¬ 

tively: 


h{x, y, z)dydz = (x) 


h{x,y, z)dxdz = 1^2 (y), and 


h{x, y, z)dxdy = h^iz). 


• Mechanism Qi is ei-private since Vi is Laplace-distributed with parameter 

• Mechanism {Qi,Q 2 ) is e 2 -private. Margining out V 3 shows that (Fi,V 2 ) ~ which 

guarantees e 2 -privacy according to Theorem [T^ 

• Mechanism {Qi,Q 2 ,Q 3 ) is ea-private. It holds that: 


d 

—P(Qim = V’l, Q 2 U = ' 02 , and Q^u = ^s) 
d 

—/l(01 -U,1p2- U, - U) 

dh{x, y, z) dh{x,y,z) dh{x, y, z) 


dx 


dy 


dz 


x=ipi—u, 

y=ip 2 -u, 

Z=1p^—u 


(38) 


Algebraic manipulation of the last expression establishes the result: 


dh dh dh 
dx ^ dy ^ dz 


fxg fyg . f 9y , f 9 , f 9z 


1 ^ ^ / w . 

1^2 '■£2 ^£2 ''e| ''£2 

-sgn(2/)e2^ - sgn(z)e3^ sgn{y)€2^ 

^62 ^€2 ^62 

-sgn{z)e3^ 

^€2 


= esh, 


(39) 


where we used the properties = -sgn{y)e 2 le 2 , fx + fy = -sgn(y)e 2 /, and gy+gz = -sgn{z)e 3 g, 
where the last two identities were derived in the proof of Theorem □ 


Additionally, performing multiple rounds of privacy relaxations can be performed in the context 
of Theorem |11| is possible. In that case. Theorem 12 is independently applied to each component. 
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Figure 4: Privacy level can be repeatedly relaxed. For each round of relaxation Ci —>■ Ci+i, the distribution 
of the next noise sample Vi+x depends only on the last noise sample Vi. Past noise samples can 

be discarded from memory, thus, there is no complexity incurred from repeatedly relaxing privacy level. 


3.3.1 Multiple Rounds of Privacy Relaxation 


Theorem states that it is possible to repeatedly use Theorem [T^ to perform multiple privacy 
level relaxations. An intuitive proof of Theorem | 12 | can be constructed by considering Scenarios 
and introduced in the beginning of Section Specifically, Theorem constructs a coupling such 
that Scenario replicates Scenario Therefore, once the first round of privacy relaxation ei —€2 
occurs, the two scenarios are indistinguishable. The second ronnd of privacy relaxation €2 —)• £3 is 
performed by starting at the first step of Scenario 


In practice, Theorem 12 allows for an efficient implementation of an arbitrary number or privacy 
relaxation rounds ei —)• 62 In particular, only the most recent privacy level and 

noise sample Vi need to be stored in memory. Sampling for V^+i depends only on current privacy 
level Cj, current noise sample Vi and next privacy level e^+i. Past privacy levels {ej}j<i, past noise 
samples {Vj}j^i, and future privacy levels {ej}j>i+i are not needed. 


3.4 A Private Stochastic Process 


Theorems 10 and 12 offer a novel dimension to the Laplace mechanism. Specifically, these results 
establish a real-valued stochastic process {I4 : e > 0 }. Sampling from the process {V'e}e>o performs 
gradual release of sensitive data for the continuum of privacy levels ( 0 ,00). Consider the mechanisms 
Qf: that respond with Q^u = = n -|- I4. Then: 


• Ve is optimally distributed, i.e. Laplace-distributed with parameter ^ 

• Any e-truncated response {y(T}o-e(o,e] is e-private. 

Samples of the process {Ve}e>o are plotted in Figure This process features properties that 
allow efficient sampling: 


• It is Markov, Vs-\-Vt\Vq, for s < g < t. Thus, a sample of the process I4 over an interval [ei, €2] 
can be extended to [ei, 63], for 63 > 62. 
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It is lazy, i.e. 14+^ = with high probability, for d <C 1. Therefore, A sample of the process 
{K} ei<e <62 can be efficiently stored; only a finite (random) number m of points 
where jumps occur need to be stored for exact re-construction of the process. 


4 Applications 

4.1 Crowdsourcing Statistics with RAPPOR 

Theorems pT|and|12|perform gradual release of private data by releasing responses that approximate 
the identity query q{u) = u. In practice, however, the end-user of private data is interested in more 
expressive queries q. The spectrum of such queries vastly varies. Examples include the mean value 
n of a collection of private data ui,...,Un, and solutions to optimization problems [8]. 

Our results are directly applicable to a broad family of queries which are approximated by private 
mechanisms built around the Laplace mechanism. Specifically, consider mechanisms based on the 
Laplace mechanism and have the form shown in Figure The database of private data is initially 
preprocessed and, then, additive Laplace-distributed noise is used. The result is post-processed in 
order to maximize the accuracy of the response. Informally stated: 

Corollary 13. Let {lL,d) be a metric space of sensitive data, y be a set of responses, and e > 0 be 
a privacy level. Let 

• F : U ^ A (M”) be a preprocessing step with sensitivity /3 that is invariant of e, 

• Eg : M" —)> A (M") be the Laplace mechanism with parameter e: 

Cf^u = u + V, where V ^ e (40) 


• Ge : M” —)■ A (T) be a post-processing step. 

Consider the e-private mechanism 

GoCoF -.U ^ A{y). 


(41) 


Then, there exists a composite mechanism that performs gradual release of sensitive data u€lL. 

Thus, our results are directly applicable to a existing privacy-aware mechanisms in e.g. smart 
grids m, m, and user’s reports [2]. On the other hand, applying our results is not yet possible 
for mechanisms that do not fulfill this assumption, such as privately solving optimization problems 
with stochastic gradient descent [8j. 

In particular, Google’s RAPPOR [2] is a mechanism that collects private data from multiple 
users for “crowdsourcing statistics” and can be expressed in terms of the Laplace mechanism. RAP¬ 
POR collects personal information from users such as the software features they use and the URLs 
they visited, and provides statistics of this information over a population of users. Algorithmically, 
a Bloom filter B is applied of size k is applied to each user’s private data u: 

R:Lf^{0,l}^ y=[yu...,yk]=B{u), (42) 


where IL is the space of private data, in particular, the set of all strings. Next, each bit yi is 
perturbed with probability / and the result is memoised: 


/ : {0,1}^ {0,1}'^, z = [zi,..., Zk] = f{y), where Zj 


0, w.p. ^a, 

< 1, w.p. ia, 

yi, w.p. 1-Cl, 


(43) 
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Figure 5: User 1 wants to share his sensitive data, such as his date of birth, in the a social network. 
Although, user 1 has no privacy concerns when sharing this information with his close friends 2 and 3, 
he has gradually increasing privacy issues for other members of the network. Specifically, a group A of 
distant users should not be able to collude and extract more information than what it is intended. 


where “w.p.” stands for “with probability” and a G [0,1] is a parameter. Finally, RAPPOR 
applies another perturbation each time a report is communicated to the server. This perturbation 
is equivalent to the map (43) but differently parametrized: 


5 : {0,1}^ ^ {0,1}'^, w = [wi,...,wk]=f{z), where F{wi = l) = <^' | J (44) 

[ 7 , if = 0 , 

where /?, 7 G [0,1] are parameters. RAPPOR’s differential privacy guarantees relax (increased e) 
for small values of a and 7 , and large values of /3. 

An important limitation of RAPPOR is that parameters a, /?, and 7 are forever fixed. However, 
there are reasons that require the ability to update these values in a way that the privacy is relaxed 
and the accuracy is increased: 


• Due to the non-trivial algorithm of decoding the reports, a tight accuracy-analysis is not 
possible. Instead, the accuracy of the system is evaluated once the system is bootstrapped^ 
Our results makes it possible to initialize the parameters with tight values a —)• 1, /3 —?• .5, 
7 —>■ .5, and subsequently relax the parameters until a desired accuracy is achieved. 


• Once a process or URL is suspected as malicious, the server would be interested in relaxing 
the privacy level and performing more accurate analysis of the potential threat. Once such 
a threat is identified, our result allows users to gradually relax their privacy parameters and 
the server can more confidently explore the potential threat. 


In order to apply Theorems 10 and 12 to RAPPOR, we express the randomized maps (43) and 
(44) using the Laplace mechanism. Specifically, consider the functions / and g that add Laplace 
noise and project the result to { 0 , 1 }: 


/(V’) 

5(0 


V’ + ^/ > 2 

C + K> 


, where Vy Lap 
111(27) 


1 


ln( 47 (l-/ 3 ))J 


—2 In a 
, where Vg Lap 


1 


-In (4/3(1 - 7 )) 


(45) 

(46) 


^Even in that case, estimating the actual accuracy can be challenging since it should be performed in a differential 
private way. 
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where tpX ^ {0,1}, Lap(6) is the Laplace distribution with parameter b, and [expr] G {0,1} is 1 
if, and only if, expr is true. Note that functions / and g have the structure of Figure Moreover, 
it can be shown that / and g applied component-wise to y and 2 ; are reformulations of the maps 
/ and g. Therefore, privacy level relaxation is achieved by sampling noises Vf and Vg as suggested 
by our results. 

4.2 Privacy in Social Networks 

The context of social networks provides another setting where gradually releasing private data is 
critical. Consider a social network as a graph G = {V, E), where V is the set of users and E the set 
of friendships between them. Each user owns a set of sensitive data that can include the date of 
birth, the number of friends and the city he currently resides. In the realm of social media, user’s 
privacy concerns scale with the distance to other users of the network. Specifically, an individual is 
willing to share his private data with his close friends without any privacy guarantees, is skeptical 
about sharing this information with friends of his friends, and is alarmed to release his sensitive 
data over the entire social network. Therefore, an individual i chooses a different privacy level ej 
for each user j G V as a decreasing function between users i and j: 

1 

'' “ WJy 

where d is a distance measure, e.g. the length of the shortest path between nodes i and j. Then, 
user i could generate an Cj-private response yj independently for each member j of the network. 
However, more private information than desired is leaked. Specifically, consider the part of the 
social network shown in Figure where user i = 1 wishes to share his sensitive data u, such as her 
date of birth. Then, consider a group A CV of users residing far away from user 1 such that the 
privacy budget ej allocated by user i to each member j of the group A is small: 

d{l,j) 1 ^ Cj <C 1. 

In the case that members of the large group A decide to collude, they can infer more information 
about the sensitive data u. Specifically, if a large group A averages the received responses {yj : 
j G A}, the exact value of sensitive data u is recovered. Indeed, composition theorem implies that 
only -privacy of sensitive data u is guaranteed. For a large group A, this privacy level 

becomes very loose. 

Our approach mitigates this issue. We assume that noisy versions of the private data are 
correlated and we design a mechanism that retains strong privacy guarantees. For real-valued 
sensitive data u, user 1 samples {v^ : e > 0} from the stochastic process {V^ : e > 0}, and responds 
to user j with yj = u + Ve^, as shown in Figure]^ In the case that a large group A of users colludes, 
they are unable to extract much more information. Specifically, such a collusion renders individual’s 
sensitive information at most (max^gyi ej)-differential private. This privacy budget is significantly 
tighter than the one derived in the naive application of differential privacy and corresponds to the 
best information that a member of the group A has. After all, if a close friend leaks sensitive 
information, it is impossible to revoke it. 

5 Open Problems 

Finally, we conjecture that gradually releasing private data can be extended to any query and is, 
therefore, an intrinsic property of differential privacy. This conjecture is a key ingredient for the 


(47) 
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Figure 6: User 1 wants to share his sensitive data, such as his date of birth, in the a social network. 
Although, user 1 has no privacy concerns when sharing this information with his close friends 2 and 3, 
he has gradually increasing privacy issues for other members of the network. Specifically, a group A of 
distant users should not be able to collude and extract more information than what it is intended. 


^6 es 64 63 62 



Figure 7: User 1 draws a single sample from the stochastic process {Ue}e>o and responds to user i with 
Ui = u + 14 ., where is the privacy level against user i. Eventually, having access to more responses 
{yi}i^A does not reveal more information about private data u than the best response maxg^ e^. 
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existence of a frictionless market of private data. In such a market, owners of private data can grad¬ 
ually agree to a rational choice of privacy level. Moreover, buying the exact private data is expected 
to be extremely costly. Instead, people may choose to buy private data in “chunks”, in the sense 
of increasing privacy budgets. We conjecture that gradually releasing sensitive data without loss 
in accuracy is feasible for a broader family of privacy-preserving mechanisms beyond mechanisms 
that approximate identity queries. This work was focused mechanisms which are defined on real 
space or sensitive data lA = MA under an .^i-norm adjacency relation, and approximate the identity 
query. 
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